ABSTRACT In the U.S., more than 600,000 knee osteoarthritis (OA)-related total knee joint replacement (TKR) cases are reported every year, exceeding $17 billion estimated direct costs annually. There is a growing need for disease- modifying therapies that prevent or delay the need for TKR. However, development of such therapies remains challenging due to the lack of objective and measurable OA biomarkers for disease progression. The course of the OA is highly variable between individuals and the OA progresses too slowly, making it difficult to identify sensitive OA biomarkers capable of capturing minor changes on the knee joint. This has slowed development of effective therapies and prevents physicians from providing the most effective advice about minimizing the need for TKR. In this project, our goal is to develop imaging biomarkers to monitor minor OA-related changes in knee joint health that lead to TKR. To achieve this goal, we will combine novel deep learning algorithms with clinical and imaging data from the Osteoarthritis Initiative (OAI). The OAI dataset includes clinical data, biospecimens, radiographs, and magnetic resonance (MR) images collected over 8 years. The proposed project has three Specific Aims: (i) to develop an automated OA-relevant biomarker identification tool from the bilateral posteroanterior fixed-flexion knee radiographs using deep convolutional neural networks (CNNs) and recurrent neural networks (RNNs) combined with the OA progression outcome of subjects (n = 882); (ii) to develop an automated OA-relevant biomarker identification tool from structural and compositional MR images using 3D CNNs with RNNs combined with the OA progression outcome of subjects (n = 882); and (iii) to determine whether deep learning?based imaging biomarkers can act as surrogates to predict the OA progression using a subject cohort (n = 296) independent of the cohort used to identify imaging biomarkers. The proposed project will couple deep learning with diagnostic radiology to unveil key combinations of OA-relevant features directly from images with minimal user interaction. This will facilitate fast individualized assessment of OA progression using whole knee joint images directly. If successful, this study will bring new insights into the development of imaging biomarkers for OA progression and more broadly into our understanding and treatment of OA. The knowledge gained in this project will help to advance close monitoring of OA progression by opening new perspectives on the regions and parameters for potential inclusion in both intervention studies and clinical practice.